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Abstract. The aim of this article is to present a smoothness criterion for 
Schubert varieties in generalized flag manifolds G/B in terms of patterns in 
root systems. We generalize Lakshmibai-Sandhya's well-known result that says 
that a Schubert variety in SL(n)/B is smooth if and only if the corresponding 
permutation avoids the patterns 3412 and 4231. Our criterion is formulated 
uniformly in general Lie theoretic terms. We define a notion of pattern in 
Weyl group elements and show that a Schubert variety is smooth (or ratio- 
nally smooth) if and only if the corresponding element of the Weyl group 
avoids a certain finite list of patterns. These forbidden patterns live only in 
root subsystems with star-shaped Dynkin diagrams. In the simply-laced case 
the list of forbidden patterns is especially simple: besides two patterns of type 
A3 that appear in Lakshmibai-Sandhya's criterion we only need one additional 
forbidden pattern of type D4. In terms of these patterns, the only difference 
between smoothness and rational smoothness is a single pattern in type B2. 
Remarkably, several other important classes of elements in Weyl groups can 
also be described in terms of forbidden patterns. For example, the fully com- 
mutative elements in Weyl groups have such a characterization. In order to 
prove our criterion we used several known results for the classical types. For 
the exceptional types, our proof is based on computer verifications. In or- 
der to conduct such a verification for the computationally challenging type 
Eg , we derived several general results on Poincare polynomials of cohomology 
rings of Schubert varieties based on parabolic decomposition, which have an 
independent interest. 



1. Introduction 

Let G be a semisimple simply-connected complex Lie group and B be a Borel 
subgroup. The generalized flag manifold G/B decomposes into a disjoint union of 
Schubert cells BwB / B, labeled by elements w of the corresponding Weyl group W . 
The Schubert varieties X w = BwB/B are the closures of the Schubert cells. A 
classical question of Schubert calculus is: For which elements w in the Weyl group 
W , is the Schubert variety X w smooth? 

This question has a particularly nice answer for G = SL{n). In this case the 
Weyl group is the symmetric group W = S n of permutations of n letters. For 
a permutation w = W\ W2 ■ ■ ■ w n in S n and another permutation a = o~\ 02 ■ ■ ■ 
in Sk, with k < n, we say that w contains the pattern a if there is a sequence 
1 < pi < ■ ■ ■ < pk < n such that w Pi > w Pj if and only if o~i > o~j for all 
1 < i < j < k. In other words, w contains the pattern a if there is a subsequence in 
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w of size k with the same relative order of elements as in a. If w does not contain 
the pattern a, then we say that w avoids the pattern a. 

Theorem 1.1. (Lakshmibai-Sandhya |19| ) For a permutation w E S n) the Schubert 
variety X w in SL(n)/B is smooth if and only if w avoids the patterns 3412 and 
4231. 

There are several general approaches to determining smoothness of Schubert va- 
rieties. See Billey and Lakshmibai for a survey of known results. Kazhdan and 
Lusztig defined a weaker condition called rational smoothness. Rational smoothness 
can be interpreted in terms of Kazhdan-Lusztig polynomials |15| . |lfi| . A Schubert 
variety is rationally smooth whenever certain Kazhdan-Lusztig polynomials are 
trivial. Kumar 18 presented smoothness and rational smoothness criteria in terms 
of the nil Hecke ring, defined in ^7] . There are many other results due to Carrell, 
Peterson, and other authors related to (rational) smoothness of the Schubert vari- 
eties. For example, according to a result of D. Peterson, see Carrell and Kuttler [Hj, 
smoothness of Schubert varieties is equivalent to rational smoothness in the case 
of a simply-laced root system. Nevertheless none of these general criteria give a 
simple efficient nonrecursive method (such as the Lakshmibai-Sandhya criterion) 
for determining if a given Schubert variety is smooth or not. Recently, Billey £Q 
presented analogues of Lakshmibai-Sandhya's theorem, for all classical types B n , 
C n , and D n . However, these constructions, including the definitions of patterns, 
depend on a particular way to represent elements in classical Wcyl groups as signed 
permutations. 

The main goals of this paper are to present a uniform approach to pattern 
avoidance in general terms of root systems and to extend the Lakshmibai-Sandhya 
criterion to the case of an arbitrary semisimple Lie group G. This approach us- 
ing root subsystems will be described in the next section. Theorem 12.21 gives a 
polynomial time algorithm for determining smoothness and rational smoothness of 
Schubert varieties in G/B in terms of root subsystems. As a consequence of the 
main theorem, we get two additional criteria for (rational) smoothness in terms 
of root systems embeddings and double parabolic factorizations (see Theorems 13. II 
and E21 ■ 

Based on the ideas of root subsystems presented in this work, Braden and the 
first author [3] refined this notion and gave a lower bound for the Kazhdan-Lusztig 
polynomials evaluated at q = 1 in terms of patterns. They also introduce a geomet- 
rical construction which identifies "pattern Schubert varieties" as torus fixed point 
components inside larger Schubert varieties. This can be used to give another proof 
of one direction of our main theorem. However, due to a delay in publication, those 
results will appear first. 

In Section [21 we formulate our smoothness criterion and describe the minimal 
lists of patterns needed to identify singular (rationally singular) Schubert varieties. 
In Sectional we present a computational improvement using root system embed- 
dings that reduces the minimal lists to just 4 patterns (3 patterns) for (rational) 
smoothness test. The difference between smoothness and rational smoothness is 
exhibited in the presence or absence of rank 2 patterns. The connection to fully 
commutative elements is described in Section 0] In Section [3] we recall several 
known characterizations of smoothness and rational smoothness from the literature 
which we will use in the proof of the main theorem. In Section we reformulate 
our main result in terms of parabolic subgroups. Then we prove two statements on 
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parabolic decomposition which will be used in the proof of Theorem 12 .21 including 
Theorem 16 . 41 which gives a criterion for factoring Poincare polynomials of Schubert 
varieties. In Sectional we give the details of the proof of the main theorem. 

2. Root subsystems and the main results 

As before, let G be a semisimple simply-connected complex Lie group with a fixed 
Borel subgroup B. Let f) be the Cartan subalgebra corresponding to a maximal 
torus contained in B. Let $ G f)* be the corresponding root system, and let 
W = W<s> be its Weyl group. The choice of B determines the subset C $ of 
positive roots. The fact that a Schubert variety X w , w <G W, in G/B is smooth 
(or rationally smooth) depends only on the pair ($> + ,w). We call such a pair 
(rationally) smooth whenever the corresponding Schubert variety is (rationally) 
smooth. The inversion set of an clement w in the Weyl group W$ is defined by 

I<s>(w) = $+ n tu(<&_) , 

where $_ = {—a | a <E <&+} is the set of negative roots. 

The following properties of inversion sets are well-known, see §1, no 6]. 

Lemma 2.1. The inversion set I$(w) uniquely determines the Weyl group element 
w £ W$. Furthermore, a subset I C <E> + in the set of positive roots is the inversion 
set Iif(w) for some w if and only if there exist a linear form h on the vector space 
t)* such that I — {a G $ + | h(a) > 0}. 

A root subsystem of $ is a subset of roots Ac$ which is equal to the intersection 
of $ with a vector subspace. Clearly, a root subsystem A is a root system itself in 
the subspace spanned by A, see §1, no 1]. It comes with the natural choice of 
positive roots A + = Afl <I> + . 

By Lemma ETT1 for any w € W<s> and any root subsystem Ac$, the set of roots 
I$(w) n A is the inversion set -Ta(c) for a unique clement a 6 Wa in the Weyl 
group of A. Let us define the flattening map /a : — > Wa by setting /a(w) = a 
where a is determined by its inversion set /a(c ) = I§iw) R A. 

Recall that a graph is called a star if it is connected and it contains a vertex 
incident with all edges. Let us say that a root system A is stellar if its Dynkin 
diagram is a star and A is not of type A\ or A%. For example, B3 is stellar but F4 
is not. Our first analogue of the Lakshmibai-Sandhya criterion can be formulated 
as follows. See also Theorems EH1 EHl ETD and Q 

Theorem 2.2. Let G be any semisimple simply- connected Lie group, B be any 
Borel subgroup, with corresponding root system $ and Weyl group W = W<&. For 
w 6 W, the Schubert variety X w C G/B is smooth (rationally smooth) if and 
only if, for every stellar root subsystem A in the pair (A + ,/a(u>)) is smooth 
(rationally smooth). 

The proof of Thcor cm !2 . 21 appears in Sectiond If A is a root subsystem in $ and 
a = /a(w), then we say that the element w in W$ contains the pattern (A + ,er). 
It follows from Theorem l2 . 21 that an element in W$, containing a non-smooth (non- 
rationally-smooth) pattern is also non-smooth (non-rationally-smooth). Another 
explanation of this fact for rational smoothness based on intersection homology 
can be found in the work of Billey and Braden mentioned above. 

Let us say that an element w avoids the pattern (A + ,cr) if w does not contain 
a pattern isomorphic to (A + ,cr). Clearly, Theorem 12.21 implies that the set of 
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(rationally) smooth elements w £ W$ can be described as the set of all elements w 
that avoid patterns of several types. Since there are finitely many types of stellar 
root systems, the list of forbidden patterns is also finite. 




Figure 1 . Dynkin diagrams of stellar root systems 

Actually, the list of stellar root systems is relatively small: B2, G2, A3, B3, 
C3, and D 4 . Figure n shows their Dynkin diagrams labeled according to standard 
conventions from |7j. In order to use Theorem 12. 21 as a (rational) smoothness test 
we need to know all non-smooth and non-rationally-smooth elements in the Weyl 
groups with stellar root systems. The following table gives the numbers of such 
elements. 



stellar type: 


B-2 


G 2 


A3 


B 3 


Cz 


D 4 


non-smooth elements: 


1 


5 


2 


20 


20 


49 


non-rationally-smooth elements: 








2 


14 


14 


49 



There are several things to notice about the table. In the simply-laced cases ^3 
and D4 the numbers of non-smooth and non-rationally-smooth elements coincide. 
The rationally smooth elements in B n are exactly the same as the rationally smooth 
elements in C n . This explains why the number 14 appears in both B3 and C3 cases. 
Note that in general the number of non-smooth elements in B n is not equal to the 
number of non-smooth elements in C n . For examples, we have 268 non-smooth 
elements in the B4 case and 270 non-smooth elements in the C4 case. 

There are exactly two non-smooth elements of type A3 — they correspond to 
the two forbidden patterns that appear Lakshmibai-Sandhya's criterion. Although 
there are 49 non-smooth elements of type D4, only one (!) of these 49 elements 
contains no forbidden ^3 patterns. These three patterns (two of type A3 and one 
of type D4) are all patterns that are needed in the case of a simply-laced root 
system (A-D-E case). 

For all stellar types, let si,S2, . . . be the simple reflections generating the cor- 
responding Weyl groups labeled as shown on Figure ^ Thus in both ^3 and D4 
cases the reflection S2 corresponds to the central node of the corresponding Dynkin 
diagram. We will write elements of corresponding Weyl groups as products of the 
generators Si. 

Theorem 2.3. Suppose that <E> is a simply-laced root system. Then the Schubert 
variety X w , w 6 W$, is smooth if and only if w avoids the following three patterns: 
two patterns of type A3 given by the elements S2S1S3S2 and S1S2S3S2S1 and one 
pattern of type D4 given by the element S2S1S3S4S2. 

Remark that, D. Peterson has shown (unpublished, see Carrell and Kuttler [5]) 
that in the simply-laced case a Schubert variety is smooth if and only if it is ra- 
tionally smooth. Thus in the previous claim we can replace the word "smooth" by 
the phrase "rationally smooth." 
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For the case of arbitrary root systems (including non-simply-laced ones), we 
need to list forbidden patterns of types B2, G2, B 3 , and C 3 . The only non-smooth 
element of B2 is S2S1S2. The non-smooth elements of type G2 are the 5 elements 
in the interval in the Bruhat order [siS2S\,w [ (w Q is excluded), where w a is the 
longest Weyl group element for type G2. There are also 6 non-smooth elements 
of type B 3 and 6 non-smooth elements of type C 3 that contain no forbidden B2 
patterns. The following theorem summarizes this data and gives the minimal list 
of patterns for the smoothness test. 

We will write [a, b, ... , c] to denote the collection of words 1, a, b, . . . , a We 
concatenate this collection with another word as follows: [a, b, c] d is a shorthand 
for the four words d, ad, bd, cd. 

Theorem 2.4. Let $ be an arbitrary root system. The Schubert variety X w , w G 
W$ , is smooth if and only if w avoids the patterns listed in the following table: 



stellar type 


forbidden patterns 


# patterns 


B 2 


S2S1S2 


1 


G 2 


[s 2 ]siS2Si[s 2 ] ; SiS 2 SiS 2 Si 


5 


A 3 


S2S1S3S2, S1S2S3S2S1 


2 


B 3 


S2S1S3S2, SiS 2 S 3 S2Si[s 3 , S3S2, S2S3, S2S3S2] 


6 


c 3 


[S3js2sls3.s2t.s3], S3S2S1S2S3, S1S2S3S2S1S3S2S3 


6 


D 4 


S2S1S3S4S2 


1 



All Weyl group elements for the types B2 and G2 are rationally smooth. Thus we 
can ignore all root subsystems of these types in rational smoothness test. Rational 
smoothness can be defined in terms of Kazhdan-Lusztig polynomials that depend 
only on the Weyl group. The Weyl groups of types B 3 and C 3 are isomorphic. 
Thus the lists of non-rationally-smooth elements are identical in these two cases. 
The following theorem presents these lists. 

Theorem 2.5. Let $ be an arbitrary root system. The Schubert variety X w , w G 
W$ , is rationally smooth if and only if w avoids the patterns listed in the following 
table: 



stellar type 


forbidden patterns 


# patterns 


A 3 


S2S1S3S2, S1S2S3S2S1 


2 


B 3 = C 3 


[S3]S2S1S3S 2 [S 3 ], [S2]S3S2S1S2S 3 [S 2 ], 
SlS2S 3 S2Si[s 3 , S 2 S 3 , S3S2, S2S3S2, S3S2S3] 


14 


D 4 


S2S1S3S4S2 


1 



A quick glance on the tables in Theorems 12.41 and 12.51 reveals that the lists 
of forbidden patterns for non-simply-laced cases are longer than the list of three 
simply-laced forbidden patterns. In Section |31 we show how to reduce the list pat- 
terns above to just the forbidden patterns of types B2, A 3 , D4 using embeddings 
of root systems. 
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3. Root System Embeddings 

In this section, we present an alternative notion of pattern avoidance in terms 
of embedded root systems. Again we can characterize smoothness and rational 
smoothness of Schubert varieties. The key advantage of this approach is that we 
reduce the minimal number of patterns to just 3 for rational smoothness and 4 for 
smoothness. While we believe this approach is useful for computational purposes, 
we suspect root subsystems are better for geometrical considerations. 

Let $ and A be two root systems in the vectors spaces U and V, respectively. An 
embedding of A into $ is a map e : A — > $ that extends to an injective linear map 
U — > V. For example, any three positive roots a, /?, 7 £ $+ define an A3 -embedding 
whenever a + fa (3 + 7 and a + (3 + 7 are all in <f> + . 

Note that inner products are not necessarily preserved by embeddings as they 
are with root subsystems. Also note that every root subsystem A in $ gives an 
embedding of A into $, but it is not true that all embeddings come from root 
subsystems. It is possible that A embeds into $ but the linear span of e(A) in 
$ contains some additional roots. Nevertheless, in the simply-laced case this can 
never happen. For simply-laced root systems, the notions of root subsystems and 
embeddings are essentially equivalent. 

We will say that a fc-tuple of positive roots (fa, . . . , (3k) in gives a 62- 
embedding, A3 -embedding, or D '4- embedding if these vectors are the images of the 
simple roots in A for an embedding A — > <I> with A of type B 2 , A3, or D4, respec- 
tively. For example, i?2-embeddings are given by pairs of positive roots (Pi, fa) 
such that both vectors fa + fa and fa + 2 fa belong to <i> + . Also A3- embeddings are 
given by triples of positive roots (fa, fa, fa) such that all vectors (3i + (3%, (3% + /?3, 
and (3i+ @2 + are roots in <I> + . 

1323 

^-embedding: A^-embedding: • 

0i+l32 /3i + 2/3 2 / \ 

Pi • • • • I3 2 P13 ^ X 024 

(3i and (3\ +2f3 2 are long roots 
(3i + /?2 and fa are short roots 

012=01, 023 = 02, 034 = 03, 
/3l3 =/9l+/3 2 , 024=02+03, 
014 =01+02+03 




Figure 2. B 2 - and A 3 -embeddings 

Figure [3 illustrates B 2 - and A 3 -embeddings. The vertices on the figure corre- 
spond to the positive roots in the image of the embedding. Here we used a (fc — 1)- 
dimensional picture in order to represent collections of fc-dimensional vectors. The 
vertices on the figure are the intersections of the lines generated by the roots with 
a certain affine hyperplane. Therefore, inversion sets are determined by half planes 
in these pictures. A similar 3-dimensional figure can be constructed for D4. Egon 
Schulte pointed out [22J that the figure can be obtained by projecting 12 vertices of 
the regular 24-cell onto a tetrahedron spanned by 4 vertices of the 24-cell. In fact, 
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it can be viewed as a model in projective 3-space, and then it is actually related to 
the half-24-cell. 

The set of positive roots <E>+ in <I> and the embedding e determines the set of 
positive roots A + = e _1 ($ + ) in A. We can extend the definition of the flattening 
map to embeddings of root systems. For an embedding e : A — > $, let us define the 
flattening map f e : W$ — > Wa by setting f e (w) — a, if the inversion set of w pulls 
back to the inversion set of a i.e. Ia(o~) = g^ 1 (I<s>{w)). According to Lemma \'2. II 
the element a is uniquely defined. 

There are five A3-embeddings into a root system $ of type B3 and seven into $ of 
type C3. Among these twelve embeddings, five are necessary to classify rationally 
singular Schubert varieties and three of these embeddings lead to false positive 
classifications of rationally smooth elements in Wc 3 ■ Therefore, we introduce the 
following definition in order to eliminate the false conditions. For an embedding 
e : A — > $, let A C $ be the root subsystem in <E> spanned by the image e(A). We 
say that an embedding e : A — > $ is proper if either A is not of type B3 , C3 or A 
is of type S3 , C3 and there exists a £?2-embedding e : B 2 — > A such that 

(1) If B 2 has basis (3\,(3 2 , then e(/?i + = e(«j) for some simple root a t 6 A. 

(2) We have e~ 1 (e(A)) = Ib 2 (s2Sis 2 ) = {Pi + fa/h + 2[3 2 ,{3 2 }. In words, 
the image of the i?2-embedding intersects the image of the A-embedding 
in exactly three roots which correspond to the inversion set of the unique 
singular Schubert variety X(s 2 s\s 2 ) of type B 2 . 

The root systems B3 and C3 each have three i^-embeddings and each of these 
embeddings corresponds to exactly one proper ^-embedding. 

For an element w in the Weyl group W$, we say that w contains an embedded 
pattern of type B 2 , ^3 , or D4 if there is a proper embedding e : A — > $ such that 

• B2 : A is of type B 2 and f e {w) — s 2 s\s 2 ] 

• A 3 : A is of type A 3 and f e (w) = sis 2 s 3 s 2 si or f e {w) = s 2 sis 3 s 2 ; 

• D4 : A is of type D4 and and f e (w) = S2S1S3S4S2. 

Recall here that the Coxeter generators Sj of Weyl groups of types B 2 , A3, and D4 
are labeled as shown on Figure ^ Note, that the reduced expressions above are all 
of the form: central node conjugated by its neighbors or neighbors conjugated by 
the central node. 

Let $ v be the root system dual to Its Weyl group W$v is naturally isomorphic 
to W$. For an element w € W$, we say that w contains a dual embedded pattern 
whenever the corresponding element in W$v ~ W$ contains an embedded pattern 
given by a proper embedding e : A — > $ v . 

Theorem 3.1. Let G be any semisimple simply- connected Lie group, B be any 
Borel subgroup, with corresponding root system $ and Weyl group W = W§ . 

(1) For w S W, the Schubert variety X w is rationally smooth if and only if w 
has no embedded patterns or dual embedded patterns of types A3 or D4 . 

(2) For w 6 W<&, the Schubert variety X w is smooth if and only if w has neither 
embedded patterns of types B 2 , A3, or D4, nor dual embedded patterns of 
types A3 or D± . 

Note that, the element w, corresponding to a smooth Schubert variety X w , may 
contain dual embedded patterns of type B 2 . Thus smoothness of Schubert varieties, 
unlike rational smoothness, is not invariant with respect to duality of root system. 



s 



SARA BILLEY AND ALEXANDER POSTNIKOV 



Proof. Any B2, A3, or Z?4-embedding spans a root subsystem whose rank must be 
at most 4. Therefore, this theorem follows directly from Theorem 12.21 bv checking 
all root systems of rank at most 4. □ 

We mention one more computational simplification in applying Theorem 13.11 
For any w € W$, there exists a hyperplane that separates the sets of roots I<s>(w) 
and $ + \ 2$(w). Figure illustrates embedded patterns of types B 2 and A3. It 
is easy to see that each of these inversion sets is determined by a half plane. The 
black vertices "•" correspond to the roots in the inversion set I<s,(w) and the white 
vertices "o" correspond to the roots outside the inversion set J$(w). 

0i +f3 2 Pi + 2/32 
ft. o • • • ft 




Figure 3. Forbidden embedded patterns of types B2 and A3 

Therefore, in order to search for embedded patterns for w of types B2, A3 and 
D4 we only need to look for pairs, triple or quadruples of the following forms: 

B2: A pair of positive roots (/3i , /?2 ) which forms the basis of a £?2-embedding 

such that (3i ^ and (i\ + P2 £ I<s>(w). 

A3: A triple of positive roots (fli, 02, P3) which forms the basis of a proper 

^-embedding such that (1) j3\2, /?34 ^ I<s>{w) and (3i4 £ or (2) 

P23 £ h{w) and [3 13 , (3 2 4 6 h{w); 
D 4 : A 4-tuple of positive roots /3 2 , Pa) which forms the basis of a _D 4 - 

embedding such that fi\ + 2f3 2 + (3 3 + /3 4 £ I<j>(w) and (3i + f3 2 + $3, P\ + 

4. Other Elements Characterized by Pattern Avoidance 

In a series of papers (see [TI], [22], [22 an d reference wherein), Fan and Stem- 
bridge have developed a theory of fully commutative elements in arbitrary Coxeter 
groups. By definition, an element in a Coxeter group is fully commutative if all its 
reduced decompositions can be obtained from each other by using only the Coxeter 
relations that involve commuting generators. 

According to the fully commutative elements in type A are exactly the permu- 
tations avoiding the pattern 321. In types B and D, Stembridge has shown that the 
fully commutative elements can again be characterized by pattern avoidance |23l 
Theorems 5.1 and 10.1] 
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We note here that fully commutative elements are easily characterized by root 
subsystems as well. The following is an unpublished theorem originally due to 
Stembridge [2"5| . 

Proposition 4.1. Let W be any Weyl group with corresponding root system 
Then w S W is fully commutative if and only if for every root subsystem A of type 
A2, B2, or G2 we have /a(w) ^ where is the unique longest element of 
Wa ■ In other words, w is fully commutative if and only if w avoids the patterns 
given by the longest elements in rank 2 irreducible root systems. 

Remark 4.2. Fan, Stembridge and Kostant also investigated abelian elements in 
Weyl groups. An element w G W is abelian if its inversion set I(w) contain no three 
roots a, [3, and a + (3. Equivalently, w G W is abelian if the Lie algebra b nw(b_) is 
abelian, where b is Borel and b_ is opposite Borel algebras. For simply-laced root 
systems, the set of abelian elements coincides with the set of fully commutative 
elements. The set of abelian elements has a simple characterization in terms of 
embedded patterns. Indeed, by definition, w £ W<s> is abelian if and only if there is 
no A 2 -embedding e : A — > $ such that the flattening f e (w) is the longest element 
w£ of W A - 



5. Criteria for Smoothness and Rational Smoothness 

In this section, we summarize the three criteria for smoothness and rational 
smoothness we rely on for the proof of Theorem 12. 21 

Let oti, . . . , a n be the simple roots in $ and let Z[fj] denote the symmetric algebra 
generated by a±, . . . , a n . For any w,v € W such that w < v, let us define K WiV 6 
Z[f)] by the recurrence 

Kw.w = ol for w — v; 

K _ f K WSuV if v < vS'i 

w > v ~ \ K WSi>v + {wsiai)K WSi>VSi if v > vsi 

for v < w and any simple reflection such that WSi < w. Then K w v is a polynomial 
of degree £(v) in the simple roots with non-negative integer coefficients. These 
polynomials first appeared in the work of Kostant and Kumar |17| on the nil Hecke 
ring, see [2] for the recurrence. 

Kumar has given very general criteria for smoothness and rational smoothness 
in terms of the nil-Hecke ring. Through a series of manipulations which were given 
in , one can obtain the following statement from Kumar's theorem for finite Weyl 
groups. Kumar's theorem in full generality applies to the Schubert varieties for any 
Kac-Moody group. However we would need to work with rational functions of the 
roots. 

Theorem 5.1. |181 Irl] Given any v,w 6 W such that v < w, the Schubert variety 
X VWo is smooth at e WWo if and only if 

(1) K w<v = H a. 

where Z(w 1 v) — {a G : v ^ s a w}. 
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We can simplify the computations in Theorem 15.11 by evaluating this identity 
at a well chosen point. The modification reduces the problem from checking a 
polynomial identity to checking degrees plus a numerical identity. Checking the 
degrees can be done with a polynomial time algorithm since this only depends on 
the number of positive roots. 

Lemma 5.2. Let r 6 f) be any regular dominant integral weight i.e. a(r) £ N+ for 
each a £ Given any v,w £ W such that v < w, the Schubert variety X VWo is 
smooth at e WWa if and only if 

(2) \Z(w,v)\=£(v) and K w , v (r) = JJ a(r). 

a£ Z (w ,v) 

Proof. We just need to prove the equivalence of (JTJ and J3J. Dyer has shown 
that K W}V is divisible by H aeZ (w,v) a - Therefore, K w , v = H aeZ (w,v) a if and onl y 
if their quotient is 1. We can check that the quotient is 1 by checking the degrees 
are equal 

t(v) = deg(K WiV ) = deg ( J] a\=\Z(w,v)\, 

(in which case the quotient is a constant) and that K w>v (r) = Y[ a £Z(w v) a ( r )- 

Remark 5.3. Note, by choosing r such that a(r) is always an integer, we do not 
have to consider potential round off errors when checking equality. 

Remark 5.4. A Schubert variety X w is smooth at every point if and only if it 
is smooth at e^. Therefore, we only need to check K Wa . WWa {r) = a(r) 

a£Z(w ,ww ) 

when \Z(w , ww a )\ — £(ww ) or equivalently \{a £ $+ : s a < w}\ = £(w). 

The next criterion due to Carrell-Peterson is for rational smoothness. The Bruhat 
graph B(w) for w € W is the graph with vertices {x 6 W : x < w} and edges 
between x and y if x = s a y for some a € where s a is the reflection corresponding 
to a, 

(v,a) 

s a v = v- — r-a. 

z(a, a) 

Note the Bruhat graph contains the Hasse diagram of the lower order ideal below 
w in Bruhat order plus some extra edges. Let 

then P w (t 2 ) is the Poincare polynomial for the cohomology ring of the Schubert 
variety X w . 

Theorem 5.5. jS] The following are equivalent: 

(1) X w is rationally smooth at every point. 

(2) The Poincare polynomial P w (t) = ^2 v < w t e ^ of X w is symmetric (palin- 
dromic). 

(3) The Bruhat graph B(w) is regular of degree £(w), i.e., every vertex in B(w) 
is incident to £(w) edges. 
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We can relate this theorem to the inversion sets 7$ (w) using the following simple 
lemma, see 0. 

Lemma 5.6. Fix a reduced expression s ai s a2 ■ ■ ■ s ap = w G W . Let f3\, ...,/?„ be 
the simple roots in $ + . The following sets are all equal to the inversion set I$(w): 

(1) $+ n io($_) 

(2) {a G $+ : s a w < w}. 

(3) {s ai s a2 ■ ■■s aj ^ 1 (3 a] : 1 <j <p} 

Let us label an edge (x, s a x) in B(w) by the root a G $+. Then, by Lemma [5^61 
the edges adjacent to w in the Bruhat graph B(w) are labeled by the elements of 
J$(w) so the degree deg(w) of the vertex w is £{w). At any other vertex x < w we 
know #{a 6 $+ : s a x < x} — £(x) so deg(x) = ^(x) + #{ct G $ + : x < s a x < w}. 
Therefore, we have the following lemma. 

Lemma 5.7. The Bruhat graph B(w) is not regular if and only if there exists an 
x < w such that 

deg(x) > deg(w) ^=> #{a G $+ | x < s a x < w} > l(w) - t(x). 

6. Parabolic Decomposition 

In the first lemma below, we give an alternative characterization of pattern 
containment in terms of a parabolic factorization. This leads to an alternative 
characterization of smooth and rationally smooth elements in the Weyl group. We 
also give a method for factoring some Poincare polynomials of Schubert varieties. 

Fix a subset J of the simple roots. Let $ J be the root subsystem spanned by 
roots in J, and let = <I> J n <&+ be its set of positive roots. Let Wj be the 
parabolic subgroup generated by the simple reflections corresponding to J. Let 
W J be the set of minimal length coset representatives for (moding out on 

the left). In other words, 

(3) W J = {v G W | v-\a) G $+ for any a G 

Every w G W has a unique parabolic decomposition as the product uv — w where 
u G Wj, v G W J and £(w) = i(u) + t(v), and conversely, every product u G Wj, 
v G W J has £(uv) = £(u) + £(v) ^ Prop. 1.10]. Equivalently, if w — uv is the 
parabolic decomposition and s ai s a2 ■ ■ ■ s Qp , Sb 1 Sb 2 ■ ■ ■ Sh are reduced expressions 
for u, v respectively then each s ai G Wj and s ai s a2 • ■ ■ s ap Sb 1 Sb 2 ■ • ■ s&. is a reduced 
expression for w. 

Let A C $be any root subsystem. It was shown in |H] that A is conjugate to Q J 
for some subset J of the simple roots, i.e. there exists a V\ G W J such that fi(A) = 
$> J . Clearly, Wa and Wj are isomorphic subgroups since the Dynkin diagrams for 
A and $> J are isomorphic as graphs. If there exist multiple isomorphisms, any one 
will suffice. 

Lemma 6.1. Let A c $ be any root subsystem. Suppose that vi(A) = <5> J for V\ G 
W J as above. Let w G W. Let u G Wj and let u' — u-j" uv\ be the corresponding 
element in Wa under the natural isomorphism. Then Q.a(w) = u' if and only if 
there exits i>2 G W J such that w — UV2- 

Proof. The element v\ G W J gives a one-to-one correspondence a i— > vi(a) between 
positive roots in A and positive roots in $> J . Also for any V2 G W J , maps 
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positive roots of <J>' 7 to positive roots inAc$ and negative roots of $ J to negative 
roots of A C $. 

Let v 2 = u~ 1 viw. We claim v 2 6 W J since for any a G <f>j, t , 2 ^ 1 (a) = 
w^ 1 Vj) 1 u(a) > 0. Then we have a bijection from the inversions of u to the in- 
versions of w in A: 

a G A + n I&(w) ur 1 (a) < 

-$=>• v 2 ~ 1 u~ 1 vi(a) < 
u~ 1 vi(a) G 
G 

□ 

Theorem 6.2. Let $ 6e any rooi system and let Ji, . . . , J s be a collection of subsets 
of simple roots such that all parabolic subsystems <£>' 71 , . . . , $ Js are stellar and they 
include all possible stellar types present in the Dynkin diagram of Then w eW 
is (rationally) smooth if and only if it cannot be presented in the form w — v^ 1 uv 2 , 
where vi,v 2 G W Ji , u G Wj it for i G {1, . . . , s}, and u is (rationally) singular 
element in Wj i . 

Proof. Suppose that A is any stellar root subsystem in Then A is conjugate to 
$' 7 , where J = Ji for some i. Now Lemma IH7T1 shows that Theorem l6.2l is equivalent 
to Theorem □ 

Each stellar parabolic subset Ji in Theorem 16 . 21 consists of a node in the Dynkin 
diagram together with its neighbors. We need to pick all nonisomorphic such sub- 
sets. For example, s = 1, for $ of type A n ; s — 2, for any other simply laced type; 
and s = 3 for $ of type B n or C n with n > 4. 

Theorem 16.21 implies the following statement. 

Corollary 6.3. Let us fix any subset of simple roots J . Suppose that u G Wj and 
vi,v 2 G W J are such that Vi 1 uv 2 is a (rationally) smooth element in W . Then u 
is a (rationally) smooth in element in Wj. 

It was shown in ^ that, for any w G W and a subset J of the simple roots, 
the parabolic subgroup Wj has a unique maximal element m(w, J) G Wj below w 
in the Bruhat order. The following theorem generalizes the factoring formulas for 
Poincare polynomials found in |12l IT] and Thm.11.23]. Using this theorem one 
can simplify the search for the palindromic Poincare polynomials which appear in 
Theorem 15.51 

Theorem 6.4. Let J be any subset of the simple roots. Assume w G W has the 
parabolic decomposition w — u ■ v with u G Wj and v G W' J and furthermore, 
u — m{w,J). Then 

p w (t) = p u (t) pT' \t) 

where P^ (t) = t^ z ^ is the Poincare polynomial for v in the quotient. 

zew J ,z<v 

Proof. Let B(w) = {x G W \ x < w} and B w .,(v) = {z G W J \ z < v}. We will 
show there exists a rank preserving bijection / : B(w) — ► B{u) x B w j{v). 

Given any x < w, say x has parabolic decomposition x = yz with respect to 
J, then y < u — m(w, J) since m(w, J) is the unique maximal element below w 
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and in Wj. Furthermore, Proctor |2()l Lemma 3.2] has shown that if z £ W is 
also a minimal length element in the coset Wjz, then z < w if and only if z < v. 
Therefore, we can define a map 

(4) / : B(w) -> B(u) x B w j(v) 

by mapping x to (y,z). Note that this map is injective and rank preserving since 
£(x) = £(y) + l(z) by the properties of the parabolic decomposition. 

Conversely, given any y £ W such that y < u and given any z £ S^j(w), then 
actually y £ Wj and we have yz < w with ^(j/z) = £(y) + £(z). Therefore, yz 
can be written as a subexpression of the reduced expression for w which is the 
concatenation of reduced expressions for u and v. Furthermore, f(yz) — (y, z) 
since z is the unique minimal length coset representative in the coset containing 
yz. Hence / is surjective. □ 

Gasharov J5| an d Billey have shown that, for the classical types, the Poincare 
polynomials of rationally smooth Schubert varieties have very nice factorizations. 
Gasharov and Reiner |18| . Ryan [23 j and Wolper [2E] have in fact shown that 
smooth Schubert varieties can be described as iterated fiber bundles over Grass- 
mannians. We see a similar phenomena for the exceptional types. 

Corollary 6.5. Every Poincare polynomial of a rationally smooth Schubert vari- 
ety in types A n , B n , C n , D n , Eq, £7, E%, G%, F4 factors into a product of symmetric 
factors each of which are Poincare polynomials indexed by elements in a maximal 
parabolic quotient W/Wj. 

We only use the following lemma in the proof of Theorem 12.21 while proving our 
criteria for rational smoothness. 

Lemma 6.6. Let w — uv be the parabolic decomposition of w 6 W with u £ Wj 
and v £ W J . 

(1) If X u is not rationally smooth then X w is not rationally smooth. 

(2) For any root subsystem A C $> J , we have J$(u)nA = J$(w)nA. Therefore, 
if u has a pattern (A + ,cr) then so does w. 

Note, it is not true that if X v is singular in the quotient G/Pj then X w is 
necessarily singular in G/B. Also, it is possible for both X u to be smooth in G/B, 
X v smooth in the quotient and yet X w to be singular in G/B if u ^ m(w, J). 

Proof. To prove the first statement, assume X u is not rationally smooth. Then 
there exits a vertex x < u in the Bruhat graph for u where deg(x) is too large by 
Theorem 15.51 and Lemma 15.71 namely 

#{a £ $+ : x < s a x < u} > £(u) - £(x). 

We claim deg(xv) is too large for X w . Note, x < s a x < u implies s a £ Wj, 
and therefore xv < s a xv < w by the properties of the parabolic decomposition. 
Therefore, 

£ : xv < s a xv < w} > ^{a £ $ + : x < s a x < u} 
> £(u)-£(x)= £{w) - £(v) - £(x) = £(w)-£(xv). 

Hence, X w is not rationally smooth, proving the first statement. 

The second statement follows directly from Lemma RTT1 □ 
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7. Proof of Theorem 12.21 



The semisimple Lie groups come in 4 series: A n , B n ,C n , D n and 5 exceptional 
types Eq, Er, Eg, F^G^. The proof of Theorem l2.2l for the four infinite series follows 
easily on the known characterization for smooth Schubert varieties in terms of 
pattern avoidance. The proof for the exceptional types was much more difficult. 
One might imagine that a routine verification would suffice for these finite Weyl 
groups. However, both verifying smoothness in F4 and rational smoothness in Eg 
directly would be impossible in our life time using previously known techniques. 
The exceptional types were proved with the aide of a large parallel computer after 
making several reductions in complexity. These reductions in complexity also give 
insight into the intricate geometry of the exceptional types. 

Recall, classical pattern avoidance is defined in terms of the following function 
which flattens any subsequence into a signed permutation. Let B n denote the 
signed permutation group. Elements in B n can be written in one-line notation as 
an ordered list of the numbers 1, ... ,n with a choice of sign for each entry. For 
example, 321 = S2S3S2S1S2 £ B n (barred numbers are negative). The group B n is 
isomorphic to the Weyl group of type B n and C n . The Weyl group of type D n is 
the subgroup of B n in which all elements have an even number of negative entries; 
and the Weyl group of type A n _i (the symmetric group S n ) is the subgroup in 
which all elements have no negative entries. 

Definition 7.1. Given any sequence a\a<i . . .a k of distinct non-zero real numbers, 
define fl(aia2 . . . a&) to be the unique element b = b\ ... bk in Bk such that 

• both a,j and bj have the same sign. 

• for all i,j, we have \bi\ < \bj\ if and only if |dj| < \aj\. 

For example, fL(6, 3, 7, 1) = 3241. Any sequence containing the subsequence 
6, 3, 7, 1 does not avoid the pattern 3241. 

Theorem 7.2. Let W be one of the groups Wa„_ 1 ,Wb„,Wc„ orWo n and 

let w € W. Then X w is (rationally) smooth if and only if for each subsequence 
1 < i\ < *2 < *3 < *4 < n, a(wi 1 Wi 2 Wi 3 Wi i ) corresponds to a (rationally) smooth 
Schubert variety. 

In order to prove Theorem 12.21 we claim ^(w^w^w^Wi^) — v if and only if 
Ia(v) = I<s>(w) n A where A is the root subsystem of type B4 in the span of 
e \w il I j e \wi 2 1 1 e \wi 3 1 1 e \wi 4 1 ■ We will prove this claim in type B, the remaining cases 
are similar. Then verification of types A3, B4, C4 and D4 suffices to check the 
theorem in the classical case. 

For type B n , let us pick the linear basis e\, . . . ,e n in f)* such that the simple 
roots are given by ei, e% — e\, . . . , e n — e n -i- Then <i> + = {ek ± ej \ 1 < j < k < 
n} U {ej : 1 < j < n}. A signed permutation w acts on R" by 




Explicitly, i$(w) = R u><!>_ is the union of the following three sets 



{w(-ej) : Wj < 0} 

{w(ej - e k ) : j < k, Wj > \w k \} 

{w(±ej - e k ) : j < k, w k < and \wj\ < \w k \}. 
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Therefore, deciding if w(—ej) or w(±ej — ek) € I®(w) depends only on the relative 
order and sign patterns on Wj and Wk- By definition of the classical flattening 
function H(wiWi 2 Wi 3 w^) = v G Wb 4 if WiWi 2 Wi 3 Wi 4 and V1D2W3V4 have the same 
relative order and sign pattern. Hence, when A is the root subsystem of type B4 
determined by e^j, e|«; i2 |, e\ Wi |> e |«;i 4 |! we have flA = I/\(v) if and only if 

fl( , u;i 1 iUj 2 UJj3«;j 4 ) = w. This proves the claim and finishes the proof of Theorem 12. 21 
for the classical types. 

Next consider the root systems of types G2 and F4 . We can simply check Theo- 
rem !2.2l bv computer using the modified version of Kumar's criterion for determining 
smoothness and the Carrell-Peterson criteria discussed in Sectional In particular, 
for G2, we use Remark 15.41 to find all singular elements Schubert varieties. They 
are X SlS2Sl , X S1828182 , X S2SlS2Sl , X S1S2818281 , X S2SlS2SlS2 (assuming a. x is the short 
simple root). Pattern avoidance using root subsystems does not offer any simpli- 
fication of this list. However, using root systems embeddings in Section |3J these 
singular Schubert varieties all follow from one B2 pattern. All Schubert varieties 
of type G2 are rationally smooth. 

For F4 we use the following algorithm to verify Theorem 12. 21 

(1) Make a matrix of size \W\ 2 containing the values K w ^ v (r) computed recur- 
sively using the formula 

K ( r \ = { K WSi . v (r) if v < vsi 

w \ K W8itV (r) + (wsiOi(r)) K W8iiV8i (r) if v > vsi 

where is any simple reflections such that wsi < w. 

(2) Identify all subsets {71, . . . , 7 P } C for p = 2, 3, 4 such that {71, . . . , 7 P } 
forms a basis for a root subsystem A of type ^2,^3,-63 or C3 (no root 
subsystems of types G2 or D4 appear in F4). Let B be the list of all such 
root subsystem bases. 

(3) For each such root subsystem A with basis B E £>, find all singular Schubert 
varieties X v using R.emark l5.4l and add Ia(v) to a list called BAD(_B). Note 
this list can be significantly simplified by removing all B^^Cs Schubert 
varieties which are classified as singular using a B2 root subsystem. 

(4) For each w S Wf 4 , check £(w) — \{"f ■ s 7 < w}\ and K Wo , WWo (r) = 
U-yez(w ww ) ^( r ) ^ an( i 0Iu y ^ no B & B exists such that I$(w) nspan(B) 
is a member of BAD(_B). 

To verify the theorem for rational smoothness in F4 we use the same algorithm 
except BAD(_B) should contain all of the inversion sets for rationally singular Schu- 
bert varieties of types ^3 , B3 , C3 , D4 and in Step 01 use the palindromic Poincare 
polynomial criterion for rational smoothness from Theorem 15. 51 

Finally, for E§ , E7 and E% it suffices to check the theorem on E$ since the 
corresponding root systems and Weyl groups are ordered by containment. Note, 
the Weyl group of type Eg has 696,729,600 elements so creating the matrix as in 
Step 1 above is not possible with the current technology. Therefore, a different 
method for verifying the main theorem was necessary. Recall, D. Peterson has 
shown, see [0], that smoothness and rational smoothness are equivalent for simply 
laced Lie groups, i.e. A n ,D n and E 6 ,Ei,E S - Unfortunately, computing P w (t) for 
all w £ E$ and applying either of the Carrell-Peterson criteria in Theorem 15.51 is 
out of the question, however we made the following observations: 
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(1) In E$, if P w (t) is not symmetric then approximately 99.989% of the time 
the only coefficients we need to check are of t 1 and t^™)- 1 . In fact, in all 
of Eg, all one ever needs to check is the first 6 coefficients (starting at t 1 ) 
equals the last 6 coefficients. Note, the coefficient of t 1 is just the number 
of distinct generators in any reduced expression for w and the coefficient 
f ft(w)-i j g .j- ne numDer f v — WSa for a £ such that £(v) = l{w) — 1 
which can be efficiently computed. 

(2) In Eg, if P w (t) is symmetric then there always exists a factorization accord- 
ing to Theorem 16 . 41 where J is a subset of all simple roots except one which 
corresponds to a leaf of the Dynkin diagram. This factored formula makes 
it easy to check the palindromic property recursively. 

(3) Let J be the set of all simple roots in Eg except ot\. Here we are labeling 
the simple roots according to the following Dynkin diagram: 



By Lemma l6~fil we only need to test w = uv where v £ W/Wj, u £ Wj ~ 
Wd 7 and X u smooth. There are 9479 elements of Wd t which correspond 
to smooth Schubert varieties and 2160 elements in W/Wj. 

With these three observations, we can complete the verification of Eg, using the 
following algorithm: 

(1) Identify all root subsystems of types A3, D4 and their bases as above (since 
Eg is simply laced it has no root subsystems of the other types). Call the 
list of bases B. 

(2) For each such root subsystem A with basis B £ B, find all singular (or 
equivalently rationally singular) Schubert varieties X v using Remark 15.41 
and add Ia(v) to a list called BAD(B). Note this list can be significantly 
shortened by removing all _D 4 Schubert varieties which are classified as 
singular using an ^3 root subsystem. 

(3) Identify all 2160 minimal length coset representatives in the quotient WjyVF 
(moding out on the left). Call this set Q. 

(4) Identify all 9479 elements in Smooth-VFj » Smooth-£> 7 := {u £ W Dl ■ 
X u is smooth} using classical pattern avoidance or root subsystems. 

(5) For each u £ Smooth-VFj and v £ Q, let w = uv and check if B £ B exists 
such that U(w) n span^) £ BAD(S). 

(a) If yes, check as many coefficients as necessary to show P w (t) is not 
symmetric. Here 5 coefficients sufficed for all u, v. 

(b) If no, attempt to factor P w (t) by taking J' to be all simple roots except 
one of the leaf nodes of the Dynkin diagram and if w = m(w, J') v' 
or w" 1 — m(w~ l , J') v' in the corresponding parabolic decomposition 
then apply Theorem 16.41 For every w in this case, there exists some 
such J' so that P w {t) = P u >{t) P^'' (t) where P$"'' ' (t) is symmetric 
and P u > (t) factored into symmetric factors recursively by peeling off 
one leaf node of the Dynkin diagram at a time. 

This completes the proof of Theorem 12.21 Theorems 12.31 12.41 and 12.51 follow 
directly. 
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Conjecture 7.3. Let <£> be a simply laced root system of rank n. Say (<$>+, w) is 
not a rationally smooth pair. We conjecture that one only need to compare the first 
n coefficients and the last n coefficients of P w (t) in order to find an asymmetry. 
Equivalently, the Kazhdan-Lusztig polynomial Pid,w(o) has a non-zero coefficient 
among the terms q 1 , q 2 , . . . , q n . 

We can show that for A n one only needs to check n — 2 coefficients, for D 5 and 
Eq one needs to check 3 coefficients, and for E$ one needs to check 5 coefficients. 
For F4 (which is not simply laced) one needs to check 3 coefficients, and for B 5 one 
needs to check 6 coefficients. 
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As with Appel and Haken's 1976 proof of the Four Color Theorem, our proof 
has met some criticism as to the merit of using a computer verification. In fact, 
it has been rejected by two journals based not on the importance or originality of 
the results, but on the method of proof. We believe quite to the contrary that 
every significant computer aided proof is a major accomplishment in expanding the 
role of computers in mathematics. It is like practicing to use induction 2000 years 
ago; it was a highly creative and influential achievement. With years of practice, 
we have become quite proficient with the induction technique. However, computer 
aided proof is a fledgling technique that certainly will have a major impact on the 
future of mathematics. Therefore, we hope our method of proof will actually make 
a much broader impact on the future of mathematics than our main theorem. 

Perhaps the only thing a complete human proof could add is an intuitive expla- 
nation for why stellar root subsystems contain all the bad patterns. This remains 
an open problem. 
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